Report

Implementation

In this lab, I simply replace the image stack input with the game state. For the model architecture, I simply using four Dense layers (unit = 512) with Relu. The information will be calculated by the last Dense layer which outputs two numerical values. The numerical values mean the Q values given the state.

The worth mention part is that, the game state is pre-processed before sending into the model. The intuition is to make the training faster.

I first turn the future pipes' absolute y location into relative location. Later, I discretize the state into buckets. (The pre-process is actually mimicing the example code in the last lab).

Insight

In this lab, I find out that DQN is really powerful. Last time when we use Q-Learning algorithm, it takes a lot of episodes for training a good bird. However, when we use DQN, it may only need a few episodes to train a good bird.

Also, compared to the image stack, giving game state is much easier for training.